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ABSTRACT 

The Thinking Skills Program of the Saginaw Public 
Schools was evaluated in a process evaluation. The School District of 
the City of Saginaw (Michigan; operates a supplemental educational 
delivery system in reading and mathematics that consists of Chapter-I 
funded elementary and secondary compensatory education (CE) . 
Elementary CE is both a push-in ''operating in regular classrooms; and 
a pull-out program that serves 1,819 students in grades 1 through 5. 
The secondary CE is a self-contained classroom program that involves 
about 621 students in grades 6 through 8. The Thinking Skills Program 
fTSP) is part of the secondary CE. Questionnaires completed by 15 
middle school teachers and 495 students, as well as interviews with 8 
principals and classroom observations of 18 teachers provided 
evaluation data. Nearly 25 percent of teachers indicated that they 
had difficulty implementing T5P methods, and participating students 
still showed low achievement, particularly in reading. 
Recommendations for program improvement centered on these findings, 
and also supported the purchase of additional computers and supplies, 
better teacher training, better program management, and better 
reporting of testing results. Two text tables and eight in the 
appendixes present study findings. The eight appendixes contain 
checklists, questionnaire results, and teacher ratings. (SLD) 
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Program Description 

The School District of the City of Saginaw operates a supplemental 
educational delivery system in reading and mathematics consisting of two 
programs — elementary and secondary Conpensatory Education (CE). The 
elementary CE is both a push-in program (that operates in the regular 
classroom) and a pull-out program (periodically taking students out of regular 
classrooms) that serves 1,819 students in grades one through five. The 
secondary CE is a self-contained classroom program which involved 
approximately 621 students in grades six through eight. The CE programs are 
funded by both the Federal Education Consolidation and Improvement Act (ECIA) 
Chapter 1 and Article 3 of the State School Aid Act. 

Summarized in the chart belcw are demographic characteristics that 
describe both the elementary and secondary levels of CE in greater detail. 
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A review of the gains/losses for the TSP program in Table 2, shows that 
for the last three years in basic and advanced reading a total of one of six 
(16.7%) composite grade comparisons had a positive gain. The mathematics area 
shows that for the last three years , a total of five of six (83.3%) composite 
grade comparisons showed a positive gain. Thus the TSP program has been much 
more successful in the area of mathematics than reading. 

If the reading and mathematics three-year gains are weighted across all 
12 comparisons , a -0.4 loss is evident. 

A review of literature was undertaken to determine if these findings were 
consistent with other TSP sites across the United States. A review of the 
articles showed more positive results in the area of mathematics. However, 
the results reported were always positive and larger than the results observed 
for our TSP program. 



Process Evaluation Procedures 

A process evaluation involves monitoring a program throughout the year to 
determine if the program is being implemented as planned. This makes it 
possible to identify strengths and weaknesses that influence a program's 
outcome. 

The process evaluation this year focused on secondary CE program in 
grades 6-8. The program is in its fourth year of implementation and termed 
locally the Thinking Skills Program (TSP) or nationally the Higher Order 
Thinking Skills (HOTS) program (see Appendix A for a checklist for middle 
school principals interested in HOTS for a further in-depth operational 
description) . 

Two questionnaires, a structured interview guide and a classroom 
observation instrument were used to gather information relative to the TSP 
program (see Append i : B for a copy of these instruments). All TSP teachers 
and their students were to be surveyed and each principal/assistant principal 
for instruction at the four middle schools were to be interviewed plus each 
compensatory education classroom teacher involved with the program were also 
to be observed. The questionnaires were distributed to teachers and students 
via interoffice mail on January 24, 1994. The completed questionnaires were 
to be returned via interoffice mail by Friday, February 11, 1994. Interviews 
with the principal and assistant principal for instruction were scheduled to 
take place February 8-10, 1994. Classroom observations of an entire classroom 
period were scheduled to start on February 9, 1994 and finish on February 18, 
1994. 
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Presentation of Process Data 



The Thinking Skills Program (TSP) Teacher Questionnaire (see Appendix B 
for a copy) was mailed through interoffice mail to the 18 TSP teachers on 
January 24, 1994 and was due back on February 11, 1994. As of torch 4, 1994 
when results were tabulated, 15 of 18 (83.3%) TSP middle school teachers had 
returned the questionnaire. The detailed tabulated results are presented in 
Appendix C* 

The TSP teachers were also sent the Thinking Skills Program (TSP) Student 
Questionnaire on January 24, 1994 (see Appendix B for a copy). The teachers 
were directed to administer the survey in their classrooms and to return the 
completed instruments on or before February 11, 1994. As of March 4, 1994 
when results were tabulated, 495 of 621 (79.7%) of the 6-8 grade students had 
completed the questionnaire. The results of these questionnaires are 
presented in Appendix D. 

The middle school principals and their assistant principals for 
instruction were interviewed starting on February 8, 1994 and concluding on 
February 10, 1994 using the Thinking Skills Program (TSP) Principal/Assistant 
Principal Interview Guide (see Appendix B for a copy). The structured 
interviews were conducted by a trained evaluator. As of March 4, 1994 when 
tabulations were completed, 8 of 8 (100.0%) of the principals had been 
interviewed. The results of the principals are presented in Appendix E. 

In addition, all 18 TSP teachers were observed by one of three evaluators 
using the 1993-94 TSP Classroom Observation Instrument (see Appendix B for a 
copy). The observations were completed from February 9-18, 1994. Each TSP 
teacher was observed over the course of a 50+ minute classroom period. A 
summary of the results of these observations can be found in Appendix H. 



What follows are the salient points stemming from this year's process 
evaluation efforts of the 1993-94 TSP Program. The manager and two evaluates 
of the Evaluation Department reviewed the results of the questionnaires/ 
observations (Appendix G) and summarized them into areas of agreemer / and 
disagreement (when multiple respondent groups were asked a similar question), 
and a single perspective (when a single respondent group was asked a question 
that was not asked of another group). These areas of agreement, disagreement, 
and a sole perspective were then categorized as strengths and weaknesses. The 
item number(s) after each strength or weakness refers to the master question 
number given in Appendix F that relates this number back to the three 
survey/interview instruments (for compensatory education teachers, students, 
and principal/assistant principals and the classroom observation, 
respectively). 

Are as of Agreement 

An examination of the questionnaires/interview guide/classroom 
observation (Appendix H) from each respondent group (Appendices C, D, and E) 
the following areas of agreement about the program were noted. These areas of 
agreement then were categorized as a strength or weakness. The areas of 
agreement indicating a strength in the program included the following. 

• Over 85% of the responding teachers (86.7%) and principals 
(87.5%) strongly agree/agree that students are interested 
and motivated by TSP activities and materials (Item 10). 

• Approximately 75% of the responding teachers (73.3%) and 
principals (75.0%) believe that TSP teacher morale and 
interest in the program is high (Item 11). 

• More than 74% of responding teachers (80.0%) and principals 
(75.0%) strongly agree/agree that the inservice in TSP 
methods was helpful for them and other teachers (Item 6). 

• More than 74% of responding principals (75.0%) and teachers 
(80.0%) strongly agree/agree that other teachers are coopera- 
tive with TSP teachers (Item 7). 

8 
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• Approximately half of the responding teachers (46.7%) and 
principals (50.0%) strongly agree/agree that students can 
write more complete questions and answers as a result of 
being in a TSP class (Item 8). In a related question, 
51.7% of the students agree that as a result of the TSP 
class they can write more complete questions and answers 
to questions (Item 22). 

• A majority of principals (75.0%) and TSP teachers (66.7%) 
perceive other regular education teachers supportive of 
TSP (Item 23). 

• All principals (100.0%) and 86.7% of the TSP teachers see 
the principals as supportive of the TSP (Item 24). 

• Students, teachers, and principals agree that one of the 
major strong points of the TSP program was the hands-on 
computer experience (Item 32). 

• More than a majority of principals (62.5%) and students 
(51.1%) would like to see the TSP "continue" next year 
(Item 37) . 



The areas of agreement indicating a weakness in the program included the 
following. 




• A minority of principals (25.0%) and teachers (33.3%) indi- 
cated that not all of the students eligible for the TSP 
program participated (Item 17). 

• A small percentage of teachers (20.0%) and principals (37.5%) 
indicated that team teaching and/or block scheduling (elements 
of the new middle school configuration) have caused interfer- 
ence with the TSP program (Item 31). 

• Approximately 75% of the TSP teachers (73.3%) and principals 
(75.0%) indicate TSP teachers have regular communication with 
other regular education teachers concerning the TSP program 
(Item 25). It is the desire of the Chapter 1 consultants 
that compensatory education teachers have frequent and mean- 
ingful conmunications with regular education teachers. 

Areas of Disagreement 

Again from an examination of the questionnaires/interview guide/classroom 
observation (Appendix H) from each respondent group (Appendices C, D, and E) , 
the following areas of disagreement about the program were noted by the 
reviewers. Ihese areas of disagreement were also categorized as either 
strengths or weaknesses. 9 



The areas of disagreement indicating a strength in the program included 
the following. 



• Vhen strongly agree/agree relative to whether students 
find discussion at the end of a unit easier is examined, a 
range of response levels with the largest being for teachers 
at 93.3% followed by principals at 75,0% and students at 
69.3% were seen (Item 2). 

• The groups differed on whether students who do well enough 
at the end of their first year should be taken out of the 
program with the strongly agree/agree being the greatest 
for principals at 62.5% follov/ed by teachers at 46.6% and 
students at 32.3% (Itana 9). [It should be noted that the 
student question was worded in the reverse and that the 
disagree/strongly disagree percent is reflected above.] 



The areas of disagreement indicating a weakness in the program included 
the following. 



• The groups differed in strongly agree/agree relative to 
whether classrooms have enough computers with the largest 
positive response from principals at 100.0% followed by 
students at 82.6% and teachers 66.7% (Item 1). 

• A total of 87.5% of the principals and 60.0% of the teachers 
felt they had enough supplies for the TSP classrooms (Item 
29). 

• All responding teachers (100.0%) strongly agree/agree that 
students are encouraged to use multiple problem-solving 
techniques; while 12.5% of the principals strongly agree/ 
agree and the remaining principals (37.5%) are unsure 
(Item 12). 

• Students shew the largest strongly agree/agree of the three 
groups at 74.7% that what they learn in the TSP class helps 
them out with what is taught in their other classes, while 
this agreement is less for both teachers (53.3%) and princ- 
ipals (62.5%) (Item 3). In addition, 76.2% of the students 
feel that what they learn in the TSP class carries over to 
the lessons in their other classes (Item 21). 

• A total of 26.7% of teachers have found difficulty in imple- 
menting TSP methods, while a total of 12.5% of the principals 
have not found TSP methods implemented (Item 18). 

• The groups differed in their agreement with whether students 
compose eight questions at the end of each unit, principals 
agreed the most at 37.5% followed by students at 10.1%, and 
then teachers at 6.7% (Item 19). 

10 
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• Responding groups differed in strongly agree/agree relative 
to whether having students write and answer eight questions 
at the end of each unit stimulates learning with teachers 
at 20.0%, principals at 59.7%, and students at 75.0% (Item 
4). It should be noted that sane teachers and students were 
told not to respond to this item because of the optional 
nature of this aspect of the TSP program. 

• Over 79% of the teachers (80.0%) and principals (87.5%) 
strongly agree/agree that displaying students' work stimu- 
lates learning, while only 55.5% of the students say their 
learnings were displayed (Item 5). 

• There was some disagreement about the weaknesses with the TSP 
program. The top weaknesses follow: 



- Teachers 

— Some poor software selections because they 
are mastered too quickly (20.0%). 

— More students than computers (13.3%). 

- Principals 

— Advanced techniques of TSP teaching need 
specialized training of staff (37.5%). 

— Advanced techniques need to be incorporated 
into regular education teachers' daily 
routine (37.5%). 

— Perception that compensatory education equals 
special education (25.0%). 

- Students 

— Teacher requires us to work (think) too much 
(6.8%). 

— Working on the computer (6.7%). 
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Areas From a Single Perspective 

Finally from an examination of the questionnaires/observation 
guide/classroom observation (Appendix H) from each respondent group 
(Appendices C, D, and E) , the following areas from the standpoint of a single 
respondent were noted by the reviewers. These areas from a sole perspective 
were categorized as either a strength or weakness. The areas of a single 
perspective indicating a strength in the program included the following, . 



• A large majority of the students (79.9%) feel their teachers 
are comfortable with TSP materials and computers (Item 14). 

• Almost all the students (88.3%) perceive their teachers as 
encouraging them to think about a problem in different ways 
(Item 15). 

• All teachers indicated they have 17 or less students per 
TSP session (Item 27). Smallest reported class sizes 
ranged from 8 to 15 students (Item 28). 

• Almost all of the TSP teachers (86.7%) feel there is ample 
space for students, computers, and materials, including 
room for discussions (Item 30). 



The areas of a single perspective indicating a weakness in the program 
included the following. 



• In the 18 teacher observations, a total of three (16.7%) 
captured a TSP teacher referring to class work in the 
regular classroom and showing hew it related to TSP class 
work (Item 47). 

• [tost TSP teachers (53.3%) feel that non-TSP teachers do 
not understand the TSP program (Item 20). 

• A total of 75% of the principals indicated that they regu- 
larly communicate with TSP teachers (Item 26). 

/ 
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Recannendations 



Based on this year's process evaluation findings and in an effort to 
improve the implementation of the TSP program in the future, the following 
recommendations are offered. 



• Program leadership should determine why approximately 25% 
of the teachers had difficulty implementing TSP methods 
and potential remedies to make future full implementation 
easier, 

• Participating students still show lew achievement levels 
especially in the area of reading. To allow the TSP program 
to better show achievement in reading, implementing optional 
program elements by TSP staff such as writing and answering 
eight questions at the end of each unit, etc, may be necessary, 

• There should be a priority given to building in monies for 
the purchase of additional computers/printers and related 
supplies in next year's budget or getting approval to purchase 
these computers/printers/etc, with carry-over funds this 
school year, 

• To help promote better understanding and communication with 
regular education teachers and principals, these staff members 
need to be allowed to practice TSP methods (meta-cognition, 
decision-making, and problem-solving) within their own class- 
rooms. An inservice session relative to these methods/tech- 
niques needs to be scheduled for all middle school staff 
members, 

• Steps that are now being planned to help increase the chances 
of all eligible students be scheduled into a TSP middle school 
classroom need to be carried out. These steps include the 
following: earlier production of updated participant listing, 
earlier completion of testing, and more prompt reporting of 
results to middle schools, etc. 
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APPENDIX A 



CHECKLIST FOR MIDDLE SCHOOL PRINCIPALS 
(Conditions Under Which the HOTS Program is Effective for Chapter 1 and LD Students) 

HOTS is a general thinking skills program designed primarily for Chapter 1 and mildly impaired 
Learning Disabled students in Grades 5-8. The thinking skills are designed to also enhance social in- 
teraction and basic skills, HOTS students are currently out-performing national averages for basic skill 
gains in reading and math, and the program has been validated by the National Diffusion Network. 

HOTS represents a new approach to compensatory education. Instead of reteaching the information the 
students did not previously learn, HOTS provides the types of thinking skills that students need to be 
able to learn content the first time it is taught in the classroom. Producing basic skill gains, however, 
requires implexrenting the program in accordance with the recommendations that follow. 

1* HOTS requires a very good teacher. A weak teacher simply cannot be successful. The 
pedagogical techniques are very sophisticated. The ideal teacher is someone who is very 
bright, energetic flexible yet organized, and who above all loves to get kids to talk. 

2. HOTS requires a good overall school improvement effort in the regular classroom. HOTS is 
designed to help a good, or improving, school get better. HOTS should not be implemented 
in a school with a weak staff, or where extensive school improvement has not already 
taken place. Since HOTS does not teach content, if the needed content is not covered in the 
regular classes, basic skills scores will not go up. This means high time-on- task, and 
quality direct instruction each day in reading and math activities aligned with test 
objectives; 

3. Proper scheduling. The HOTS program is designed to substitute for, and replace , the 
remedial activities in the school. It needs a minimum of 35 minutes of instruction a day, 4 
days a week, on an ongoing basis for 1 i — 2 years. This can be done either as a pull out or as 
a separate course. Schools that want to raise math scores can optionally use the fifth day, 
or 10-15 minutes at the end of each period, for computerized math drill and practice. 

Students should ideally be kept in the program for 1 i— 2 years, even if they test out at the 
end of the first year. This extra service is legal and helps students automate their new 
problem solving skills. First and second year HOTS students should be in separate sections. 

Students should be put into HOTS at the lowest grade level in the school (or when they 
first arrive). HOTS can be implemented either with a limited number of students, or as a 
school-wide model serving all needy students at the lowest grade level. 

A teacher can handle up to about 10 students at a time with 9 Apple II computers. A teacher 
and aide can handle up to about 16 students at a time with 13-15 computers.* Other pupil- 
teacher ratios with various combinations of personnel can be considered. HOTS project staff 
will assist in identifying other possible combinations. 

♦ (It's possible to do the program in the first year with a few less computers, but only for a 
year.) 

4. Quality, classroom instruction available to HOTS students. It is citical that HOTS 
students get good content instruction in reading and math in their regular classes 
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5. Prope budgeting. Cos* include a) purchasing the needed equipment from local vendors 
and b) training and support costs. The training and support costs per school are as follows! 

Number nf ^ 



Up to 25 


25-85 


More than 85 


$750 


$900 


$1100 


300 


400 


600 


50 


50 


100 



6. 



First year * 
Second year 
Thereafter 

♦ Includes the support fee for the school and training one HOTS teacher and 
aide. Each additional HOTS teacher adds $450 to the first year costs. (No 
added cost the second year). 

The support fee includes the curriculum, phone support, the HOTSTUFF 
newsletter, videotapes, and updates for as long as you use the program. 

General support by the principal. There are a number of general leadership activities that 
increase the effectiveness of the program. The most important leadership activity is to 
implement and monitor a good overall school effectiveness program. It is also important to 
support the HOTS teacher who will have to work very hard, particularly the first year 
when the curriculum and techniques are unfamiliar. Additional support needs include: 

a ) HOTS linkage activities consist of HOTS students writing eight questions and answers 
around a block of content every three weeks. These questions and answers are then 
brought to the HOTS lab and entered into the computer to make games and quizzes 
based on the content Content teachers interested in working with the students on their 
wnting of the questions in their class should be identified and encouraged to work with 
the HOTS students. 

b) Schedule presentations about HOTS for the entire staff early in the school year This 
includes a 15 minute video overview of the program, and a 1 i-2 hour workshop to 
tiain content area teachers on how to help students write questions. The latter should 
be conducted within the first three months by the HOTS t eacher. 

0 Support public display of the HOTS students' prowess. 

7. Evaluating HOTS Instruction. DO NOT USE EH EVALUATION TECHNIQUES HOTS 
lessons are different. The best measure of the HOTS teacher's effectiveness is the number of 
complete answers he/she obtains from students-*s opposed to one word answers- without 
giving obvious hints. The more one-word answers or hints, the weaker the lesson. There 
should be little talk by the teacher, and a lot by the students. 

DO NOT WORRY IF EARLY IN A UNIT STUDENTS SEEM CONFUSED ABOUT HOW 
TO PROCEED. Learning to use textual information to deal with uncertainty is one of the 
key skills that HOTS develops. The students will be successful by the end of the unit 

Feel free to contact Dr. Stanley Pogrow if you have further questions. Dr. Pogrow can be reached 
University of Arizona, College of Education, Tucson A2 85721 or at (602) 621-1305. 
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Table A-l 



Count of Program Participants* for the Cortpensatory Education Program, 
1993-94 



Building 


K 


1 


2 


3 


4 


5 


Total 


E. Baillie 


0 


30 


19 


19 


5 


11 


84 


Coulter 


0 


2 


22 


9 


1 


6 


40 


Emerson 


0 


28 


27 


24 


9 


18 


106 


Fuerbringer 


0 


9 


22 


18 


3 


8 


60 


N. Haley 


0 


10 


22 


2 


0 


0 


34 


Hand ley 


0 


0 


0 


0 


0 


0 


0 


Heavenrich 


0 


15 


22 


29 


12 


16 


94 


Herig 


0 


12 


26 


12 


9 


10 


69 


Houghton 


0 


23 


11 


16 


2 


4 


56 


Jercme 


0 


9 


16 


7 


3 


3 


38 


J^nes 


0 


30 


•12 


12 


0 


0 


54 


Kempton 


0 


6 


18 


12 


4 


3 


43 


Longfellow 


0 


29 


26 


47 


15 


9 


126 


Longstreet 


0 


24 


22 


14 


15 


9 


84 


J. Loomis 


0 


44 


51 


25 


13 


10 


143 


M. Park 


0 


15 


22 


22 


7 


20 


86 


C. Miller 


0 


5 


16 


5 


0 


0 


26 


J. Moore 


0 


14 


20 


18 


2 


8 


62 


Morley 


0 


10 


17 


20 


4 


19 


70 


J. Rouse 


0 


8 


16 


14 


5 


10 


53 


Sal ina 


0 


14 


16 


19 


5 


1 


55 


Stone 


0 


18 


17 


24 


14 


11 


84 


Webber Elem. 


0 


41 


40 


44 


6 


14 


145 


Zilwaukee 


0 


1 


1 


4 


0 


1 


7 


TOTAL 


0 


397 


481 


416 


134 


191 


1,619 
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*Count as of January 13, 1994 computer run that included all 
participants, ^ r><| 
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Table A-2 

Gaunt of Program Participants* for the Compensatory Education 
Program, 1993-94 



Building 


n 

i 


8 


9 


Total 


Central Middle 


73 


63 


53 


189 


North Middle 


39 


36 


23 


98 


South Middle 


41 


43 


59 


143 


Vfebber Middle 


60 


74 


57 


191 


TOTAL 


213 


216 


192 


621 


*Count as of January 


13, 1994 computer run 


that 


included all 



participants. 
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Table A-3 

Count of Program Participants* for the Chapter 1 Program, 1993-94 



Building 


K 


1 


2 


3 


4 


5 


Total 


E. Baillie 


0 


30 


19 


19 


5 


11 


84 


Coulter 


0 


2 


22 


9 


i 


6 


40 


Emerson 


0 


28 


27 


24 


9 


18 


106 


Fuerbringer 


0 


9 


22 


18 


3 


8 


60 


N. Haley 


0 


10 


22 


2 


0 


0 


34 


Handley 


0 


0 


0 


0 


0 


0 


0 


Heavenrich 


0 


15 


22 


29 


12 


16 


94 


Herig 


0 


12 


26 


12 


9 


10 


69 


Houghton 


0 


23 


11 


16 


2 


4 


56 


Jerome 


0 


9 


16 


7 


3 


3 


38 


Jones 


0 


30 


12 


12 


0 


0 


54 


Kempton 


0 


6 


18 


12 


4 


3 


43 


Longfellow 


0 


29 


26 


47 


15 


9 


126 


Longstreet 


0 


24 


22 


14 


15 


9 


84 


J. Loomis 


0 


44 


51 


25 


13 


10 


143 


M. Park 


0 


15 


22 


22 


7 


20 


86 


C. Miller 


0 


5 


16 


5 


0 


0 


26 


J. Moore 


0 


14 


20 


18 


2 


8 


62 


Morley 


0 


10 


17 


20 


4 


19 


70 


J. Rouse 


0 


8 


16 


14 


5 


10 


53 


Salina 


0 


14 


16 


19 


5 


1 


55 


Stone 


0 


18 


17 


24 


14 


11 


84 


Webber Elem. 


0 


41 


40 


44 


6 


14 


145 


Z ilwaukee 


0 


1 


1 


4 


0 


1 


7 


TOTAL 


0 


397 


481 


416 


134 


191 


1,619 




*Count as of January 13 , 1994 computer run that included all 
participants* 

19 28 
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Appendix A-4 

Count of Program Participants* in the Chapter 1 Program, 1993-94 



Building 


7 


8 


9 


Total 


Central Middle 


73 


63 


53 


189 


North Middle 


39 


36 


23 


98 


South Middle 


41 


43 


59 


143 


Webber Middle 


60 


74 


57 


191 


TOTAL 


213 


216 


192 


621 



*Count as of January 13, 1994 computer run that included all 
participants. 
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Table A-5 

Gaunt of Program Participants* in the Article 3 Program, 1993-94 



Building 


K 


1 


2 


3 


4 


5 


Total 


P Ra 1 1 1 i ca 


u 




iy 


iy 


0 


11 


79 


Cr\i 1 1 hor 


u 


z 


zz 


y 


u 


6 


39 


CJUtJL bUI 1 


U 




27 


24 


0 


18 


97 


cuerDri ny sir 


U 


y 


oo 
ZZ 


18 


0 


8 


57 


IN* ncULtry 


U 


lu 


ZZ 


2 


0 


0 


34 


nctriUXGy 


U 


U 


0 


0 


0 


0 


0 




u 


lb 


ZZ 


29 


0 


16 


82 


nt;L ±y 


U 


12 


ZD 


i o 

12 


0 


10 


60 


^/"N T"S 

nuuyi luon 


U 


z J 


11 


16 


0 


4 


54 




U 


9 


16 


7 


0 


3 


35 


Jones 


U 


o n 


12 


12 


0 


0 


54 


t\t?rupL.on 


U 


/- 
D 


18 


12 


0 


3 


39 


LiOnyiieixow 


U 


29 


26 


A "J 

47 


0 


9 


111 


r C 4"" VQQ 4"* 

l/ji iy o ui. u 


U 


24 


ZZ 


1 A 

14 


u 


9 


69 


«j • LOCfni s 


r\ 
U 


44 


5l 


25 


0 


10 


130 


M Da v~l- 


U 


ID 


ZZ 


ZZ 


0 


20 


79 


P Mi' 1 1 or 


U 


D 


lb 




0 


0 


26 




u 




zU 


lo 


u 


8 


60 




u 


1U 


1 / 


20 


0 


19 


66 


U • KOUSS 


U 


o 
0 


16 


T A 

14 


0 


10 


48 


Dct jl i nd 


U 


1 A 

14 


Id 


19 


0 


1 


50 


Stone 


0 


18 


17 


24 


0 


11 


70 


Vfebber Elem. 


0 


41 


4^ 


44 


0 


14 


139 


Zilwaukee 


0 


1 


1 


4 


0 


1 


7 


TOTAL 


0 


397 


481 


416 


0 


191 


1,485 
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*Count of January 13, 1994 computer run that included all participants. 

21 28 
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Table A-6 

Count of Program Participants* for the Article 3 Program, 1993-94 



Building 


7 


8 


9 


Total 


Central Middle 


73 


63 


53 


139 


North Middle 


39 


36 


23 


98 


South Middle 


41 


43 


59 


143 


Webber Middle 


60 


74 


57 


191 


TOTAL 


213 


216 


192 


621 



*Count as of January 13 , 1994 computer run that included all 
participants. 
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SCHOOL DISTRICT OF THE CITY OF SAGINAW 
DEPARTMENT OF EVALUATION, TESTING AND RESEARCH 



TO: Thinking Skills Program Teachers 

FROM: Richard N. Claus 

RE: Thinking Skills Program Questionnaire 

DATE: January 24, 1994 



Enclosed, please find a copy of the "Thinking Skills Program (TSP) 
Teacher Questionnaire". It is part of the midyear/process review 
.of the Chapter 1/ Article 3 program and is designed to gain input 
from TSP teachers regarding the program's operation and ways it 
may be improved. 

Please take a few moments to read and answer the questionnaire as 
fully as you can. Although we ask for your name and building, 
please be assured that all responses will be kept confidential and 
only groujped responses will be reported. 

Also enclosed are copies of the "Thinking Skills Program (TSP) 
Student Questionnaire". It is also part of the process review and 
is designed to gain student input about the program. 

There are enough copies enclosed for you to have each of your 
students cctrplete one. Please have all of them do that during a 
TSP class session, asking them not to put their names on the forms 
and assuring them that all of their answers will be kept 
confidential. Also, please have all the students complete the 
form on the same day. 

Please return the completed questionnaires through interoffice 
mail to Dr. Richard Claus in the Evaluation Department by Friday , 
February 11 , 1994 . 

If you have any questions, you may contact me at Extension 307. 



RNC/gal 
Enclosures 
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Name: 



School: 

Date: 

PACT I 
DIRECTIONS : 



APPENDIX B 
THINKING SKILLS PROGRAM (TSP) 

TEACHER QUESTIONNAIRE 



Please indicate, based on your experiences with the TSP program 
this year, whether you agree with each of the following state- 
ments about your TSP classes. Circle SA if you Strongly Agree, 
A if you Agree, U if you are Unsure, D if you Disagree, or SD 
if you Strongly Disagree. 

Strongly Strongly 
Agree Agree Unsure Disagree Disagree 



The classroom has enough 
computers. 



SA 



SD 



2. As TSP units are almost com- 
pleted most students are better 
at beginning discussion. 

3. TSP activities and materials 
supplement what the students 
are taught in other classes. 

4. Having the students write and 
answer eight questions vorks 
to stimulate learning. 

5. Displaying the students' work 
stimulates learning. 

6. The inservice training in TSP 
methods was helpful. 

7. The other teachers are cooper- 
ative with TSP teachers. 

8. Students can write more complete 
questions and answers as a 
result of being in a TSP class. 

9. Students who do good enough 
at the end of their first year 
should be taken cut of the 
program. 

10. Students are interested and 
motivated by the TSP activities 
and materials. 

11. TSP teacher morale and interest 
in the program is high. 



SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 

SA \ U D . SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 



12. Students are encouraged to use 

multiple problem-solving tech- SA A U D SD 

^ niques (for example, verbal 24 

jlj^jjQ deductive reasoning, etc.) Jl 
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PART II 

DIRECTIONS: Please answer the following questions as completely as possible. 

13. What documentation, if any, did you receive to help you in determining 
which students were eligible for the TSP program? 



14. When did you receive this documentation? 
From whom? 



15. Are all of the students at your building, who are at or below the 25th 
percentile (or 36th NCE; on CAT 5 Vocabulary and/or Math Computation) 
identified for participation in the TSP program? 

Yes 

No 

If no, please specify the following: 

The number of students (estimate) who should be in the 

program but who are not. 

The number of students (estimate) who should not be in 

the program but who are. 



What do you think caused these discrepancies? 



What do you believe could be done to improve the selection process? 



25 
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16 • Have you found the TSP methods difficult to implement? 



Yes 

No 



If yes, in what ways? 

If no, why do you think the methods implement easily? 



17, Do you have students compose eight questions at the end of each unit? 



Yes 

No 



If yes, does it seem to work well and why? 
If no, why not? 



18. Do most other non-TSP teachers understand the program? 

Yes 

No 

If yes, does the understanding allow them to help your students 
in their classes? 

If no, what should be done to bring about understanding? 



19 . Are the other non-TSP teachers supportive o£ TSP? 

Yes 

No 



If yes, in what ways? 

If no, are they in opposition or indifferent to TSP? 



ERLC 
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20. Is (Are) the principal (s) supportive of TSP? 



Yes 

No 

If yes, in what ways? 

If no, are they in opposition or indifferent to TSP? 



21. Is there regular communication between you and the other teachers? 

Yes 

No 



If yes, in what ways? 

If no, ho/ do you think communication can be enhanced? 



22. How many children are in your largest TSP class? 

23. How many children are in your smallest TSP class? 



24. Do you have enough computers and supplies for your TSP classroom needs? 

Yes 

No 



If no, what are you lacking and hew much more would you need 
to consider the classroom adequately supplied? 



25. Is there ample space for students, computers, and materials including 
room tor discussion? 

Yes 

Mo 



If no, what is the space concern? 



ERLC 
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26# Since this is our first year in the middle school configuration, have 
there been problems with team teaching or block scheduling that have 
caused interference with the TSP program? 

Yes 

No 

If yes/ what are the problems and how have they caused 
interference? 



27. What do you think are the strong points of the TSP program? 



28. What do you think are the weak points of the TSP program? 



29. What do you think are some of the unexpected outcomes of the TSP program 



30. How can the program be improved? 



31. What additional supportive resources; for example, computer support 
personnel, inservice training, new software, etc.; are needed to 
help make the TSP program more successful? 



THANK YOU FOR YOUR TIME! 28 
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THINKING SKILLS PROGRAM (ISP) 
STUDENT QUESTIONNAIRE 

School: 

Date: 



PART I 

DIRECTIONS: Please indicate whether you agree with each of the following 

statements about your TSP classroom. Circle SA if you Strongly 
Agree, A if you Agree, U if you are Unsure, D if you Disagree, 
or SD if you Strongly Disagree. 

Strongly Strongly 

Agree Agree Unsure Disagree Disagree 

1. I have access to computers 

when I need it. SA A U D SD 

2. At the end of the TSP unit, 

I find it easy to begin SA A U D SD 

discuss ions . 



3. tihat I learn in my TSP 
helps me out with what is 
taught in my other classes. 

4. Writing and answering the 
eight questions helps me 

to understand the unit better. 

5. My vork from class is dis- 
played by my teacher. 

6. If I do good enough at the 
end of my first year, I should 
be allowed to stay in the 
program. 

7. My teacher is ccmfortable 
with the TSP materials and 
computers. 

8. My teacher encourages me to 
think about a problem in 
different ways. 



SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 



ERLC 



29 



APPENDIX B 



PART II 

DIRECTIONS: Please answer the following questions as completely as possible. 

9. Do you make up eight questions at the end o£ each unit? 

Yes 

No 

If yes, does it seem to help you and your fellcw students understand 

the lessons and why? 

If no, why aren't you writing them? 



10. Does what you learn in your TSP class carry over to what you learn in 
your other classes? 



Yes 

No 



If yes, in what ways? 

If no, what is different? 



11. Do you think you can write more complete questions and answers as a 
result of being in your TSP class? 

Yes 

No 



If yes, in what ways? 
If no, why not? 



ERLC 
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12 • VJhat do you think are the good points of your TSP class? 



13. What don't you like about your TSP class? 



14. Would you like to take another class like TSP next semester or year? 

Yes 

No 



15. Vhat letter grade did you get in TSP for the last six weeks? 

A 

B 

C 

D 

E 



16. How often were you absent from the TSP class during the last six week 
period? 

No absences 

1-3 absences 

4-6 absences 

7-9 absences 

10 or more absences 



THANK YOU FOR YOUR TIME! 
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THINKING SKILLS PROGRAM (TSP) 
PRINCIPAL/ASSISTANT PRINCIPAL 
INTERVIEW GUIDE 

Name: 
School: 

Date: 



DIRECTIONS: Read aloud the statements and questions below. 
Fill in answers as appropriate. 



Read Aloud: As part of the 1993-94 mid-year/process review of the 
Chapter 1/Article 3 program, an interview of the 
administrative middle school staff is being undertaken. 
The interview is designed to gain your input regarding 
the Thinking Skills Program (TSP) operation in your 
building and how it may be improved. 

PART I 

Read Aloud: First I will read a series of statenents about the TSP program operations. 

Please indicate, based on your classroom observations, whether you agreo 
with each of the following statements about your TSP classrooms. Iridic* t-- 
if you Strongly Agree, Agree, are Unsure, Disagree, or if you Strongly 
Disagree, with the statement. Any consents about your rating would be 
apprec iated. 

Strongly Strongly 

Agree Agree Unsure Disagree Disagree Comments 

1. The classrooms have enough 

computers. SA A U D SD 



2. As TSP units are almost com- 
pleted most students begin 
discussion easily. 

3. TSP activities and materials 
supplement what the students 
are taught in other classes. 

4. Having the students write and 
answer eight questions works 
to stimulate learning. 

5. Displaying the students' work 
stimulates learning. 

6. The inservice training in TSP 
methods was helpful for both 
the TSP and regular education 
teachers. 

7. The regular education teachers 
are cooperative with TSP 
teachers. 



SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 

SA A U D SD 
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PART I (Continued) 



Strongly Strongly 

Agree Agree Unsure Disagree Disagree Comnents 



8. Students can write more complete 

questions and answers as a SA A U D SD 

result of being in a TSP class. 



- 9. Students who do good enough 

at the end of their first year 
should be taken out of the 
program. 

10. Students are interested and 
motivated by TSP activities 
and materials. 

11. TSP teacher morale and interest 
in the program is high. 



SA A U D SD 



SA A U D SD 



SA A U D SD 



12. Students are encouraged to use 

multiple problem-solving tech- SA A U D SD 

niques (for example, verbal 
deductive reasoning, etc.). 



PAL- II 

Read Aloud: Now let us move onto some other operational questions about 
the TSP program. 

13. What documentation did you receive to help you in determining which 
students were eligible for the TSP program? 



14. When did you need this documentation? 

When did you receive this documentation and from whan? 
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15. Are all of the students at your building, who are at or below the 25th 
percentile (or 36th NCE; on LAT 5 Vocabulary and/or Math Conputation) 
identified for participation in the TSP program? 



Yes 
No 



If no, please specify the following: 

The number of students (estimate) who should be in the 

program but who are not. 

The number of students (estimate) who should not be in 

the program but who are. 



What do you think caused these discrepancies? 



What do you believe could be done to improve the selection process? 



16. Have you found the TSP methods implemented? 



Yes 
No 



If yes, in what ways? 

If no, why do you think the methods are not implemented? 
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17. Do the students each compose eight questions at the end of each unit? 

Yes 

Mo 

If yes, does it seem to work well and why? 

If only in some, why there and not at all sites? 

If no, why not? 



18, Are the other regular education teachers supportive of TSP? 

Yes 

No 



If yes, in what ways? 

If no, are they in opposition or indifferent to TSP? 



19. In what ways are you supportive of TSP? 



20. Is there regular communication between the TSP and the regular education 
teachers? 

Yes 

No 



ERIC 



If yes, in what ways? 

If no, hew do you think carcnunicatioo can be enhanced? 
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21. Is there regular conmunication between you and the TSP teachers? 

Yes 

Mo 



If yes, in what ways? 

If no, how do you think conmunication can be enhanced? 



22. Do you have supplies for the TSP classroom needs? 



Yes 
No 



If no, what are you lacking and how much more would you need 
to consider the classroom adequately supplied? 



23. What additional supportive resources (for exanple, computer support 
personnel, additional inservice training, new software, etc.) are 
needed to help make the TSP program more successful? 



24. Since this is our first year in the middle school configuration, have 
there been problems with team teaching and/or block scheduling that 
have caused interference with the TSP program? 

Yes 

No 

If yes, what are the problems with team teaching and/or block 
scheduling that have caused interference with the TSP program? 
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25. Vhat do you think are the strong points of the TSP program? 



26. What do you think are the weak points of the TSP program? 



27. What do you think were some of the unexpected outcomes of the TSP program? 



28, Is there another, but different way, to deliver better compensatory 
education services? 



Yes 
No 



If yes, what are the other possible delivery systems? 

If no, why is TSP superior to the other possible delivery systems? 



29. Do you want the TSP program to continue? (Check one) 

Continue 

Expand 

Terminate 

Please explain. 



Read Aloud: 

THANK YOU FOR YOUR TIME! 
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1993-94 TSP CLASSROOM OBSERVATION INSTRUMENT 
INSTRUCTIONS: Please check or fill-in as appropriate. 

Date: 

Time Class Began: 



Evaluator: Teacher: Location: 

Enrollment: Attendance: Number Percent 



I. CLASSROOM ENVIRONMENT 

1. What is the number of computers which are in good working condition? 

2. How many samples of student work are displayed on the walls? 

3. How is attendance taken? 



4. Does the teacher encourage students to solve problems with "thinking strategies"? 
Yes No 

5. Do the students seem motivated and/or enthusiastic toward the class activities? 
Yes No 

6. Does the teacher seem motivated and/or enthusiastic toward the class activities? 
Yes No 

7. Does the teacher seem to be able to motivate and/or encourage students? 
Yes No 

8. Does the teacher refer to class work in the regular classroom and shew how its 
related to TSP class work? 

Yes No 

9. Is the class reaching the end of a unit? Yes No 

If yes, are students writing a group of questions? Yes No 

38 
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II. INSTRUCTION 

1. What is the major learning activity during each ten minutes of the TSP class 
and what role does the teacher play in that activity? 



Time 



Activity 



Role 




2. What type of lesson was presented? (Please provide descriptive details.) 



ERIC 
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THINKING SKILLS PROGRAM (TSP) 
TEACHER QUESTIONNAIRE 
(N=15)* 



PART I 



DIRECTIONS: Please indicate, based on your experiences with the TSP program 
this year, whether you agree with each of the following state- 
ments about your TSP classes. Circle SA if you Strongly Agree, 
A if you Agree, U if you are Unsure, D if you Disagree, or SD 
if you Strongly Disagree. 

Strongly Strongly No 

Agree Agree Unsure Disagree Disagree Response. 

1. The classroom has enough 5(33.3) 5(33.3) 0( 0.0) 1( 6.7) 3(20.0) 1( 6.7) 
computers . 

2. As TSP units are almost com- 3(20.0) 7(73.3) 0( 0.0) 1( 6.7) 0( 0.0) 0( 0.0) 
pleted most students are better 

at beginning discussion. 

3. TSP activities and materials 2(13.3) 6(40.0) 4(26.7) 2(13.3) 1( 6.7) 0( 0.0) 
supplement what the students 

are taught in other classes. 

4. Having the students write and 1( 6.7) 2(13.3) 5(33.3) 1( 6.7) 0( 0.0) 6(40.0) 
answer eight questions works 

to stimulate learning. 

5. Displaying the students' work 7(46.7) 5(33.3) 3(20.0) 0( 0.0) 0( 0.0) 0( 0.0) 
stimulates learning. 

6. The inservice training in TSP 7(46.7) 5(33.3) 1( 6.7) 2(13.3) 0( 0.0) 0( 0.0) 
methods was helpful. 

7. The other teachers are cooper- 4(26.7) 8(53.3) 2(13.3) 0( 0.0) 0( 0.0) 1( 6.7) 
ative with TSP teachers. 

8. Students can write more complete 

questions and answers as a 3(20.0) 4(26.7) 4(26.7) 3(20.0) 0( 0.0) 1( 6.7) 
result of being in a TSP class. 

9. Students who do good enough 2(13.3) 5(33.3) 2(13.3) 4(26.7) 1( 6.7) 1( 6.7) 
at the end of their first year 

should be taken out of the 
program. 

10. Students are interested and 7(46.7) 6(40.0) 0( 0.0) 2(13.3) 0( 0.0) 0( 0.0) 
motivated by the TSP activities 

and materials. 

11. TSP teacher morale and interest 

in the program is high. 5(33.3) 6(40.0) 2(13.3) 2(13.3) 0( 0.0) 0(0.0) 

12. Students are encouraged to use 

multiple problem-solving tech- 7(46.7) 8(53.3) 0( 0.0) 0( 0.0) 0( 0.0) 0( 0.0) 
niques (for example, verbal 
deductive reasoning, etc.) 

0 40 47 
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APPENDIX C 



PART II 

DIRECTIONS: Please answer the following questions as completely as possible. 

13. What documentation, if any, did you receive to help you in determining 
which students were eligible for the TSP program? 

• CAT test scores 7 46.7 

• None, the Counseling Office did the 6 40.0 

selection process. 

• I received data sheets on sane of my 1 6.7 

students about eligibility, but did 

not receive data sheets on every student. 

• I made it a point to check the city-wide 1 6.7 

printout. 



14. When did you receive this documentation? 



• 


September 


3 


20.0 


• 


About 1-1/2 months into the school year/ 


2 


13.3 




end of the first marking period. 






• 


Second week of school 


1 


6.7 


• 


About 3 weeks after the start of school 


1 


6.7 


• 


At the beginning of school 


1 


6.7 


• 


At a TSP meeting 


1 


6.7 


• 


No response 


4 


26.7 


• 


Not applicable 


2 


13.3 



From whom? 



School counselor 7 46.7 

Assistant principal 4 26.7 

No response 2 13.3 

Not applicable 2 13.3 
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15. Are all of the students at your building, who are at or below the 25th 
percentile (or 36th NCE; on CAT 5 Vocabulary and/or Math Coirputation) 
identified for participation in the TSP program? 



Yes 
No 

No opinion 
Unsure 
Don't know 



5 
5 
3 
1 
1 



33.3 
33.3 
20.0 
6.7 
6.7 



If no, please specify the following: 

The number of students (estimate) who should be in the 
program but who are not. 



10 

50 - 60 
100 



1 
1 
1 



33.3 
33.3 
33.3 



The number of students (estimate) who should not be in 
the program but who are. 



1 

5 

2-3 



1 
1 
1 



33.3 
33.3 
33.3 



What do you think caused these discrepancies? 

• Scheduling conflicts 3 20.0 

• Students who are enrolled in 1 6.7 

CAS or have made honor roll 
were excluded. 

• Cannot estimate false positives 1 6.7 

and negatives without having 
access to all their scores. 

• Transfer of students between 1 6.7 

buildings/lack of records. 

• Late documentation and insuf- 1 6.7 

ficient class openings to 
accommodate all students. 

• No response 8 53.3 



What do you believe could be done to improve the selection process? 



ERIC 



Past year/TSP teacher input 
Earlier access to test scores 
Test scores fran core teachers 
Grades should be considered 
interview [students] to see 
how they respond to each 
teacher. 
Specific policy directive fran 
a higher authority/better book- 
keeping process and documentation. 
No response 
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3 
2 
1 
1 
1 
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20.0 
13.3 
6.7 
6.7 
6.7 
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16. Have you found the TSP methods difficult to implement? 

Yes 3 20.0 

No 10 66.7 

No opinion 1 6.7 

Yes/No 1 6.7 

If yes, in what ways? 

• Not so much with grades 6 and 7 1 6.7 

but 8th grade has been challenging 
(responded yes/no) . 

• Some of the questions in discussion 1 6.7 

are confusing. 

• The dialog and sane of the materials 1 6.7 

we are required to teach frcm are 
not designed to meet the needs of 
Saginaw students, per se. 

• I am finding it difficult to keep up 1 6.7 

with the curriculum schedule. 

If no, why do you think the methods inplement easily? 

• Because of the training we received 2 13.3 

before teaching in the program. 

• The methods implement easily due to 2 13.3 

the guide. 

• Probing questions have to [begin 1 6.7 

early in the lesson] or the students 
do not think critically. 

• Depends on the teacher's attitude. 1 6.7 

• Smaller classes - more individualized 
instruction. 1 6.7 
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17. Do you have students compose eight questions at the end of each unit? 



Yes 
No 

No opinion 
Don't know 
Yes/No 



1 6.7 
10 66.7 

2 13.3 
1 6.7 
1 6.7 



If yes, does it seem to work well and why? 

• Depending on the unit and the topic - 1 6.7 

sometimes it works well, sometiines 
not (responded yes/no). 

• We do have questions. I also have 1 6.7 

the students write summaries, comments, 
make predictions, and strategies. I 
vary the different activities so that 
the lessons are more interesting. 



If no, why not? 



ERLC 



I was not aware of that part of the 3 20.0 

program. 

It was never stressed as a necessary 1 6.7 

part of the program. 
We use to do it but now it f s a hassle 1 6.7 

to get them to do it and for them to 

understand why it's needed. 
Verbal questioning works best with my 1 6.7 

students. 

Eight questions are too many - 3 or 4 1 6.7 

is (sic) enough. 
We never used this (sic) technique. 1 6.7 



44 51 



APPENDIX C 



18 • Do most other non-TSP teachers understand the program? 

Yes 6 40,0 

No 8 53.3 

Don't knew 1 6.7 

If yes, does the understanding allow them to help your students 
in their classes? 

• Not at this time. 1 6.7 

• Yes 1 6.7 

/ • The sixth grade teachers ... understand 1 6.7 

the program . . . [and] use sane tech- 
niques in their classes to help [TSP] 
students . 

• Most other teachers may or may not 1 6.7 

believe that the socratic method is 
the best way to teach. 

• By focusing on [the students] areas of 1 6.7 

difficulty. 

If no, what should be done to bring about understanding? 

• Some form of inservice/M-5 inservice. 2 13.3 

• Periodic visitations to the classrooms. 2 13.3 

• A comprehensive training workshop 1 6.7 

involving the entire staff. 

• A short introduction at the beginning 1 6.7 

of the year would be sufficient. 

• Mini demonstration to the staff. 1 6.7 

Other? 

• I don't knew - haven't discussed it. 1 6.7 
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19. Are the other non-TSP teachers supportive of TSP? 

Yes 10 66.7 

No 1 6.7 

No opinion 4 26.7 

If yes, in what ways? 

• They supply students' vocabulary words 2 13.3 

and information for activities; share 
information about students. 

• They know we are trying to help kids, 1 6.7 

perhaps not how we do it. 

• Identify areas of weaknesses and 1 6.7 
strengths. 

• They come to me occasionally to discuss 1 6.7 

a student I have helped. 

• Those who oppose do not see the effect 1 6.7 

of what we are attempting to teach 
reach their classrooms — due to 
behavior/discipline problems in TSP 
classes. 

• Students are allowed extra time to 1 6.7 

complete assignments. 

• The sixth grade teachers ... understand 1 6.7 

the program ... [and] use some tech- 
niques in their classes to help [TSP] 
students. 

• After explanation, they knew about the 1 6.7 

class and its reason. 

If no, are they in opposition or indifferent to TSP? 

• Indifferent 1 6.7 



Other? 



My TSP students are also in my core 1 6.7 

classes . 

I haven't heard one way or another. 1 6.7 

Most do not know what this course 1 6.7 

is about. 
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20. Is (Are) the principal (s) supportive of TSP? 

Yes 13 86*7 

No 0 0.0 

No opinion 2 13.3 

If yes, in what ways? 

• Information, materials, and computers 2 13.3 

are given? rooms have been revamped. 

• [They] ... provide timely schedules, 1 6.7 

a good working environment, and 
sufficient supplies for our classes. 

• Compliment the program/visit the class- 1 6.7 

room. 

• By scheduling students: 15 into each 2 13.3 

class, not scheduling other classes 
into TSP classroom, and scheduling 
[same level] classes back to back, 
[lessening] setting up and taking 
down [time] . 

• The principal allcws the TSP teachers 1 6.7 

to attend inservices, meetings, etc. 
They try to help in any way possible. 
Vfe have always had good support [here] . 

• By understanding the program - allowing 1 6.7 

teachers to teach classes. 

If no, are they in opposition or indifferent to TSP? 

• I have no idea, they haven't approached 1 6.7 

me about the class. 
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21. Is there regular communication between you and the other teachers? 



Yes 11 73.3 

No 3 20.0 

No opinion 1 6.7 

If yes, in what ways? 

• Discussion in faculty/team meetings. 3 20.0 

• In reference to students' progress 2 13.3 

achievement, and behavior. 

• Other TSP teachers except 2, [discuss] 1 6.7 

where in the program [my class] should 
be, how to get there, etc. 

• Students are identified who can help 1 6.7 

other teachers. 

• The team teacher and I plan our TSP 1 6.7 

activity together. 

• Daily and weekly conversion, 1 6.7 

• Share ideas/materials for improvement; 1 6.7 

solve problems dealing with software 
and/or lessons. 

If no, how do you think communication can be enhanced? 

• By putting TSP students with one team 1 6.7 

of teachers and attaching the TSP 
teacher to that team. 

• [Freeing] teacher schedules 1 6.7 

• Not enough time; constraints prevent 1 6.7 

much of this from happening. 

Other? 

» Children are in my core class 1 6.7 

22. Hew many children are in your largest TSP class? 

11 2 13.3 

12 0 0.0 

13 1 6.7 

14 3 20.0 

15 7 46.7 

16 1 6.7 

17 1 6.7 

23. How many children are in your smallest TSP class? 

8 1 6.7 

9 1 6.7 

10 0 0.0 

11 2 13.3 

12 2 13.3 

13 1 6.7 

14 2 13.3 

15 55 1 6.7 
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24. Do you have enough computers and supplies for your TSP classroom needs? 

Yes 9 60.0 

No 6 40.0 

If no, what are you lacking and how much more would you need 
to consider the classroom adequately supplied? 

v 

• One compute^ for each student; one for 2 13.3 

the teacher. 

• I have 12 computers and 15 students; 1 6.7 

some software is okay, some is (sic) 
in very short supply. 

• Administration will put too many stu- 1 6.7 

dents in the program, therefore, not 
enough computers are on hand. Also 
lack of good, quality teaching conputer 
programs and textbooks/workbooks, 
ribbons. 

• Four computers with printers and one 1 6.7 

extra printer for a conputer that 
came without one. 

• Would like to order several programs 1 6.7 

for variety. 

• However, it would be great to have at 1 6.7 

least one Macintosh conputer in the 
. classrocm with all Apples. By so 

/ doing, (can gain the experience 

needed to operate both as well as be 
able to know software for both 
(responded yes). 



56 



O 49 

ERIC 



APPENDIX C 



25. Is there ample space for students, computers/ and materials including 
room for discussion? 

Yes 13 86.7 

No 2 13.3 

If no, what is the space concern? 

• Not in my room. Vfe have the discussion 1 6.7 

here, then move upstairs to use the 
computers . 

• Students are easily distracted with the 1 6.7 

work from the other class. 

• However, this year due to my advisory 1 6.7 

class, 21 students' desks reduced 
the space causing the room to look 
crowded (responded yes). 

26. Since this is our first year in the middle school configuration, have 
there been problems with team teaching or block scheduling that have 
caused interference with the TSP program? 

Yes 3 20.0 

No 10 66.7 

No opinion - 2 13.3 

If yes, what are the problems and how have they caused 
interference? 

• Too many to list and no one to tell 1 6.7 

the problems to who would LISTEN! i ! 

• Sixth grade lunch hour causes difficulty 1 6.7 

in scheduling. 

• Many students are cross scheduled between 1 6.7 

different teams. 

• You should talk with [the assistant 1 6.7 

principal here] (responded no comment). 

• We haven't had a problem yet. However 1 6.7 

I forsee a time when there will be a 
very big problem (responded no comment). 

• In the Fall, we had problems with the 1 6.7 

amount of space available. We later 
opened another computer room, 
(responded no) . 

• Sessions for each level are not back to 1 6.7 

back. I have to [reorganize five times 
daily which] eliminates individual time 
with students (responded no). 
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27. What do you think are the strong points of the TSP program? 



• New curriculum using socratic method 6 40.0 

of teaching/question and answer 





methods , and class discussion. 






• 


Use of computers/computer assisted 6 




40.0 




instruction. 






• 


Key thinking skills (meta cognition, .6 




40.0 




inference from context, decon textual- 








ization, and synthesis of information). 






• 


Lew class size 


4 


26.7 


• 


Fun while learning 


1 


6.7 


• 


Individualized instruction 


1 


6.7 


• 


Wait time 


1 


6.7 


• 


Focus on success 


1 


6.7 


• 


Focus on key vocabulary words 


1 


6.7 


• 


Monthly teacher meetings and training 


1 


6.7 




sessions 






• 


The teachers 


1 


6.7 


• 


It helps to build the students' confidence 


1 


6.7 


• 


[Teachers] can change to different 


1 


6.7 



supportive materials if needed. 



28. What do you think are the weak points of the TSP program? 



• Some of the software were poor selections/ 


3 


20.0 


too easily mastered. 






• More students than computers/not enough 


2 


13.3 


computers . 






• I was not trained well enough, to be able 


1 


6.7 


to offer enough help to the students. 






• Not enough math materials. 


1 


6.7 


• t-forking with eighth grade. 


1 


6.7 


• Not sure how the students will carry new 


1 


6.7 


skills to pass the CAT test. 






• Too many to mention. 


1 


6.7 


• The number of teachers teaching TSP. 


1 


6.7 


• It's not accessible to more students who 


1 


6.7 


would benefit from it. 






• One game called SNAP did not interest the 


1 


6.7 


students . 






• Teachers do not follow the program. Vfe 


1 


6.7 


should know there will be 2 to 3 years 






of TSP and at what grade level. 






• Trying to make the students understand 


1 


6.7 



the values of the program. 
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29. What do you think are some of the unexpected outcomes of the TSP program? 



• 


A reasonable level of computer literacy/ 


3 


20.0 




computer interest. 






• 


The students get to know each other well/ 


2 


13.3 




learn cooperation and team work. 






• 


Levels/rates of learning vary from year 


1 


6.7 




to year. 






• 


Some students begin to believe in them- 


1 


6.7 




selves. 






• 


Some students that (sic) need assistance 


1 


6.7 




aren't getting it. 






• 


Every student wants to get in. 


1 


6.7 


• 


Close relationship between student and 


1 


6.7 




teacher because of the low class size. 






• 


Some typing experience. 


1 


6.7 


• 


Scientific method type thinking 


1 


a "7 


• 


A program for problem solving 


1 


6.7 


• 


I have not participated in the program 


1 


6.7 




long enough to make an assessment. 






• 


Unsure of what [the question] means 


1 


6.7 


• 


None 


1 


6.7 


• 


No response 


2 


13.3 



30. How can the program be improved? 

• More training/inservices for new teachers 2 13.3 

in the program. 

• Include teacher input/listen to your TSP 2 13.3 

teachers . 

• Sometimes the dialog is repetitive/ 2 13.3 

immature; the students would like to 
move along a little faster. 

• Include math for those students who have 1 6.7 

that as their weak area. 

• Develop math curriculum over summer and 1 6.7 

implement it in the fall. 

• Supply more materials for math concepts. 1 6.7 

• There needs to be more linkage to the 1 6.7 

[regular education] classroom work being 
done by students. 

• If students were placed on one team, it 1 6.7 

would be beneficial. 

• By having a conputer for each student and 1 6.7 

each teacher. 

• Offer more sections to be able to include 1 6.7 

more students. 

• Someone is needed to take care of equipment, 1 6.7 

order materials, and coordinate materials. 

• Constantly reinforce the importance of 1 6.7 

following the program (HOTS) design. None 
of our teachers have the time to develop 
an alternative curriculum. 

• Continue to have monthly meetings for 1 6.7 

sharing, problem solving, and personal 
improvement. 

• No response. 52 2 13.3 
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help make the TSP program more successful? 

• New software representing grade/interest 3 20 0 

as well as linking concepts for year two. 

• Once a month training for new teachers 2 13 3 

and update training for old timers/ 
training - not a talking session. 

• More copies of the programs/software. 2 13 3 

• All of the above. f x ^'^ 

• I need for my classroom [structure - i % n 

painting and sink] to be finished. i ' 

• We need monthly meetings to share ideas, 1 

lend support, and brainstorm. i 

• No response. ^ 



6.7 
6.7 
33.3 



53 60 



APPENDIX D 

THINKING SKILLS PROGRAM (TSP) 
STUDENT QUESTIONNAIRE 
(N=495) 

part i 

or SD if you Strongly Disagree. 

Strongly Strongly 
^Xe Agree Unsure Disagree Disagree 

1. I have access to computers 245(47.8) 171(34.8) 46( 9.3) 22( 4.5) 8( 1.6) 
when I need it. 

in7r?l 232(47.4) 95(19.4) 38( 7.8) 17( 3*5) 

2. At the end of the TSP unit, 107(21.9) ZWkt.<*> 

I find it easy to begin 
discussions. 

(nlw Tqp 163(33.2) 204(41.5) 51(10.4) 56(11.4) 17 ( 3.5) 

3. What I learn m my TSP _ iOJ ^ J ' ' 
helps me out with what is 

taught in my other classes. 

' • fho 43(22 7) 70(37.0) 44(23.3) 23(12.2) 9( 4.8) 

4. Writing and answering the 4Jl^./j 

eight questions helps me a 
to understand the unit better. 

5 . * ^ fron class Is dis- 119(24.4) 205,42.!, 94(19.3) 48^9.9) 21( 4.3, 
played by ray teacher. 

6. if , do good enough at the 145(29.5, 128(26.0, 60(12.2, 83(16.9, 76(15.4, 
end of my first year, I should 

be allowed to stay in the 
program. 

7. * teacher is comfortable 206(42.0, 186(37.9) 83,16.9, IK 2.2) 5( 1.0) 
with the TSP materials and 

computers. 

9dlf48 7) 196(39.6) 32 ( 6.5) 16 ( 3.2) 10( 2.0) 

8. My teacher encourages me to 241(48./) i»oij».oj 

think about a problem in 
different ways. 

"^^entTwere directed by their teachers not to respond to this question. 



54 



61 



APPENDIX D 



PART II 

DIRECTIONS: Please answer the following questions as completely as possible, 

9. Do you make up eight questions at the end of each unit? 

Yes 50 10.1 

No 174 35.2 

No response* 271 54.7 



10. Does what you learn in your TSP class carry over to what you learn in 
your other classes? ' 

Yes 377 76.2 

No 105 21.2 

No response 13 2.6 



11. Do you think you can write more complete questions and answers as a 
result of being in your TSP class? 

Yes 256 51.7 

No 108 21.8 

No response 131 26.5 



*Many students were directed by their teachers not to respond to this question. 
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• 
• 



• 



12. What do you think are,, the good points of your TSP class? 

You get to work on conputers and learn how to use computers 211 

Writing, listening, and note taking 76 

Help fran the teacher 70 

Learning different things 51 

Playing games 37 

Working together 29 

The class is fun 24 

Problem solving skills, making my own decisions, and/or 23 

thinking for myself 

Doing math 16 

Smaller class size 13 

Reading 9 

Everything % 

It's helping me to bring up my grades 7 

None or don't know 6 

The grading y 5 

My classmates 3 

Projects 2 

Teacher values 2 

Learning about the world and nations/facts 2 

Watching movies 2 

Helps me to learn reading and math 2 
Work harder in other classes 
No homework 

Wbn't have to take the computer class next year 
No ditto sheets 
Drawing 
Respect 

You don't have to take gym 
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13. What don't you like about your TSP class? 



• I like everything 160 

• Teacher talks too much, the way the teacher treats us, and/or 34 

the teacher doesn't have enough time for each student 

• Working on the computer 33 

• The class is boring, I don't like it, no fun, the class is 26 

too easy, or sometimes it's not fun 

• Doing the written work 22 

• Not enough time on the conputer and/or we have to share 20 

computers 

• Can't get comfortable in the classroom 20 

• Math or math day 16 

• Reading assignments 15 

• Too many steps to go through for assignments, amount of time 14 

to do projects, and/or having a folder 

• Discussions and/or talking in front of the class 11 

• Some of the students 10 

• Can't take other elect ives 9 

• The class time too long or too short 8 

• We don't get a day off, we work all the time, or we work 7 

too much 

• Interruptions in our class 7 

• I don't like anything 5 

• You,. have to take TSP for two years 5 

• Rules 4 

• The grading system 3 

• The conputer games are for kids 2 

• People think the class is Special Education 1 2 

• Students can move around anytime they want to or playing around 2 

• Conputer wrk can be confusing 2 

• Working with maps 2 

• I feel I don't need TSP 2 

• Doing work that doesn't pertain to TSP 1 

• Use of the writer 1 

• They don't help 1 

• Films/movies 1 

• Can't do things in TSP that can be done in other classes 1 

• Things don't always come in handy 1 

• Not enough students in the class 1 

• We don't learn much 1 

• Me should be able to do more articulate things 1 

• No field trips 1 

• Too much thinking 1 

• The time of class 1 



ERIC 



64 

57 



APPENDIX D 



14. Would you like to take another class like TSP next semester or year? 

Yes 253 51.1 

No 209 42.2 

No response 33 6.7 



15- What letter grade did you get in TSP for the last six weeks? 

A 166 33.5 

B 177 35.8 

C 89 18.0 

D 25 5.0 

E 7 1.4 

No response 31 6.3 



16. How often were you absent from the TSP class during the last six week 
period? 

No absences 84 17.0 

1-3 absences 221 44.6 

4-6 absences 103 20.8 

7-9 absences 23 4.6 

10 or more absences 37 7.5 

No response 27 5.5 



58 Do 



APPENDIX E 



THINKING SKILLS PROGRAM (TSP) 
PRINCIPAI/ASSISTANT PRINCIPAL 
INTERVIEW GUIDE (N = 8) 

DIRECTIONS: Read aloud the statements and questions below. 
Fill in answers as appropriate. 

Read Aloud: As part of the 1993-94 mid-year/process review of the 
Chapter 1/Article 3 program, an interview of the 
administrative middle school staff is being undertaken. 
The interview is designed to gain your input regarding 
the Thinking Skills Program (TSP) operation in your 
building and how it may be improved, 

PART I 

Read Aloud: First I will read a series of statements about the TSP program operations. 

Please indicate, based on your classroom observations, whether you agree 
with each of the following statements about your TSP classrooms. Indicate 
if you Strongly Agree, Agree, are Unsure, Disagree, or if you Strongly 
Disagree, with the statement. Any comments about your rating would be 
appreciated. 

Strongly Strongly 



1. The classrooms have enough 
computers. 

2. As TSP units are almost com- 
pleted most students begin 
discussion easily. 

3. TSP activities and materials 
supplement what the students 
are taught in other classes. 

4. Having the students write and 
answer eight questions works 
to stimulate learning. 

5. Displaying the students' work 
stimulates learning. 

6. The inservice training in TSP 
methods was helpful for both 
the TSP and regular education 
teachers . 

7. The regular education teachers 

are cooperative with TSP 4(50.0) 2(25.0) 0( 0.0) 2(25.0) 0( 0.0) 

teachers . 



Agree Agree Unsure Disagree Disagree Comments 
3(37.5) 5(62.5) 0( 0.0) 0( 0.0) 0( 0.0) 
0( 0.0) 6(75.0) 1(12.5) 1(12.5) 0( 0.0) 



2(25.0) 3(37.5) 2(25.0) 1(12.5) 



prob 

1(12.5)- 5(62.5) 2(25.0) 0( 0.0) 0( 0.0) 
5(62.5) 2(25.0) 1(12.5) 0( 0.0) 0( 0.0) 



TSP goes well beyon 
regular classroom 

nr n n\ lnt0 meta-cognition 
u ^ U * U} decision making, an 
problem solvina. 



3(37.5) 3(37.5) 1(12.5) 1(12.5) 0( 0.0) JegSl^tel^ 



Two SA respondents 
disagree with 
regular teachers. 
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PART I (Continued) 



Strongly 



Stxongly 



Agree Agree Unsure Disagree Disagree Comments 



8. Students can write more complete 

questions and answers as a 0( 0.0) 4(50.0) 4(50.0) 0( 0.0) 0( 0.0) 
result of being in a TSP class. 

9. Students who do good enough 



Two respondents 
agree when "good" 



at the end of their first year 2(25.0) 3(37.5) 2(25.0) 1(12.5) 0( 0.0) Is changed to 
should be taken out of the we 
program. 



10. Students are interested and soeciai" interest 

motivated by TSP activities 6(75.0) 1(12.5) 0( 0.0) 1(12.5) 0( 0.0) JJJSIre is Jour 
and materials. 



Strongly agree ff 
special in1 
software is 
reference. 



11. TSP teacher morale and interest 

in the program is high. 4(50.0) 2(25.0) 2(25.0) 0( 0.0) 0( 0.0) 



12. Students are encouraged to use 
multiple problem-solving tech- 
niques (for example, verbal 1(12.5) 4(50.0) 3(37.5) 
deductive reasoning, etc.). 



0( 0.0) 0( 0.0) 



PACT II 

Read Aloud: 



Now let us move onto some other operational questions about 
the TSP program. 



13. What documentation did you receive to help you in determining which 
students were eligible for the TSP program? 



• CAT scores ^ 

• List of eligible students 

• Tracking list in August, 1993 

• Transfer records 

14. When did you need this documentation? 



• Early June, 1993 

• May/June, 1993 

• Early part of Summer, 

• Mid August, 1993 



1993 



4 (50.0) 

2 (25.0) 

1 (12.5) 

1 (12.5) 



3 (37.5) 
2 (25.0) 
2 (25.0) 
1 (12.5) 



When did you receive this documentation and from whom? 



When? 



• Later part of August, 
.# May/June, 1993 

• Early Summer, 1993 

• No idea 



1993 



4 (50.0) 

2 (25.0) 

1 (12.5) 

1 (12.5) 
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15. Are all of the students at your building, who are at or below the 25th 
percentile (or 36th NCE? on CAT 5 Vocabulary and/or Math Confutation) 
identified for participation in the TSP program? 

Yes 6 (75.0) 

No 2 (25.0) 

If no, please specify the following: 

The number of students (estimate) who should be in the 
program but who are not. 

• 12.5 1 (50.0) 
m 21 1 (50.0) 

The number of students (estimate) who should not be in 
the program but who are. 

• 0 1 (50.0) 

• Some 1 (50.0) 

What do you think caused these discrepancies? 

• Two separate lists (tracking and eligibility) 1 (50.0) 
didn't agree. 

• Not enough space to take all students. 1 (50.0) 

What do you believe could be done to improve the selection process? 

• Provide lists on time and make sure they agree. 1 (50.0) 

• Determine eligibility on some additional basis 1 (50.0) 
beyond CAT test scores. 



16. Have you found the TSP methods implemented? 

Yes 7 (87.5) 

No 1 (12.5) 

If yes, in what ways? 

• Guidelines followed 3 (37.5) 

• Hands-on student experiences 2 (25.0) 

• On-going training, follow-up discussions, 1 (12.5) 
and summer remediations 

• Lesson plans 1 (12.5) 

• One-on-one instruction 1 (12.5) 

If no, why do you think the methods are not implemented? 

• Lack of adequate teaching training 1 (50.0) 

• Some teachers still lecture 1 (50.0) 
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17. Do the students each compose eight questions at the end of each unit? 

Yes 3 (37.5) 

No 0 ( 0.0) 

Unsure 5 (50.0) 

If yes, does it seen to work well and why? 

If only in some, why there and not at all sites? 

• Conditioning to follow directions 1 (33.3) 
m Deepens learning when you think 1 (33.3) 

about it 

• Saw Mrs. Williams use it. It helps 1 (33.3) 
with the MEAP 

If no, why not? 



18. Are the other regular education teachers supportive of TSP? 

Yes 6 (75.0) 

No 2 (25.0) 

If yes, in what ways of teachers/instructional planning? 

• Teaming of teachers/instructional 2 (25.0) 
planning 

• Core teachers see it as part of the 1 (12.5) 
curriculum 

• No suggestion of relectance n 1 (12.5) 

• See increased student motivation 1 (12.5) 

• Teachers help to identify potential 1 (12.5) 
compensatory education students 

If no, are they in opposition or indifferent to TSP? 

• Indifferent 2 (100.0) 

• Opposition (one yes respondent) 1 ( 12.5) 

19. In what ways are you supportive of TSP? 

• Fills TSP slots in scheduling 3 

• Attends quickly to equipment and material 3 
supply needs 

• Provides advanced planning to ireet program 2 
needs 

• Recognizes positive impacts of TSP 2 
(especially in reading) 

• Allowed assistant principal to work toward 2 
meeting needs 

• Supports methods and content 2 

• Arranged for computer lab work after school 1 

• Sensitive to TSP class size limits 1 

• Talked positively about the TSP program 1 

• Viewed it as an essential part of the core 1 
curriculum 

• Support teachers going to teaches* training 1 
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20 • Is there regular communication between the TSP and the regular education 
teachers? 



Yes 6 (75.0) 

No 2 (25.0) 

% If yes, in what ways? 

• Teaming 5 

• School improvement processes 2 

• House meetings 1 

• Building meetings 1 

• Homework sensitive 1 

• Camion preparation time 1 



If no, hew do you think communication can be enhanced? 

• Include TSP in monthly staff meeting agenda 2 



21. Is there regular communication between you and the TSP teachers? 

Yes 6 (75.0) 

No 2 (25.0) 

If yes, in what ways? 

• Daily visitations 1 

• Memos 1 

• "Stump the Chump" party 1 

• Use of machinery and software 1 

• Scheduling 1 

• Individual meetings with staff as needed 1 

• Staff meetings 1 

• Team meetings 1 

• House meetings 1 

• Parent conferences 1 

• No formal arrangement 1 

• Tracking helps 1 

• Home school aides who keep track of 1 
tutoring in the evening 



If no, how do you think conmunication can be enhanced? 

• Principal to make himself more available 1 
to meet with TSP 

22. Do you have supplies for the TSP classroom needs? 

Yes 7 (87.5) 

No 

Unsure 1 (12.5) 

If no, what are you lacking and hew much more would you need 
to consider the classroom adequately supplied? 
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23. What additional supportive resources (for example, computer support 
personnel, additional inservice training, new software, etc.) are 
needed to help make the TSP program more successful? 

• None 4 (50.0) 

• TSP inservices 1 (12.5) 

• More modeling of "socratic method" 1 (12.5) 

• Additional counseling resources for 1 (12.5) 
at-risk students 

• Move software dealing with problem 1 (12.5) 
solving, conflict resolution, and 

higher order thinking 



24. Since this is our first year in the middle school configuration, have 
there been problems with team teaching and/or block scheduling that 
have caused interference with the TSP program? 

Yes 3 (37.5) 

No 5 (62.5) 

If yes, what are the' problems with team teaching and/or block 
scheduling that have caused interference with the TSP program? 

• Cross teaming 2 

• Block scheduling 2 

• Not enough slots to schedule 1 
all students 

• Third hour round robin 1 



25. What do you think are the strong points of the TSP program? 

• Hands-on experience with computers 4 

• Instructional strategies (questioning, conflict 3 
resolution, and problem solving) fit with different 
learning styles 

• Motivation for students 2 

• Smaller class size 2 

• Excellent hardware and software support 2 

• Carry over of teacher training to regular education 1 
staff 

• Student belief that they are productive 1 
o Fewer discipline problems 1 

• Less failing of at-risk students 1 

26. What do you think are the weak points of the ToP program? 

• Theory not advanced correctly; a need for training 3 
in advanced techniques 

• Regular education teachers do not use the advanced 3 
techniques of TSP teachers - an articulation concern 

« Perception that compensatory education equals 2 
special education 

• Chapter 1 requirements for participation/identification 1 

• Too short in duration (only 49 minutes/day) 1 

• None 1 
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27. What do you think were some of the unexpected outcomes of the TSP program? 



• None 2 

• Reduces discipline problems 1 

• Labeling of students 1 

• Lack of articulation of techniques 1 

• Student self-esteem 1 

• Computer literate students 1 

• Strong central office control of program doesn't 1 
alia/ for building improvement plans 1 

• Zapped for not following guidelines because of 1 
scheduling problems related to middle school 
transition 

• Kids flunk TSP 1 



28. Is there another, but different way, to deliver better compensatory 
education services? 

Yes 4 (50.0) 

No 1 4 (50.0) 

If yes, what are the other possible delivery systems? 

• Implement core curriculum with no failures 1 

• Allcw school people more input for student 1 
participation 

• Building input/autonomy to serve kids the best 1 

If no, why is TSP superior to the other possible delivery systems? 

• No idea 2 

• Opposed to secondary pull-out 1 

• Strong skill component to retrain for needed 1 
skills 

• Interdisciplinary teaming and co-teaching 1 
(infusion) 

. • Reading progress is "OK" 1 

29. Do you want the TSP program to continue? (Check one) 

Continue 5 (62.5) 

Expand 1 (12.5) 

Terminate 2 (25.0) 

Please explain. 

- Continue 

• More work related to skill development 1 

• Should allow for interdisciplinary teaming 1 
and co- teaching 

- Expand 

• Poor implementation 1 

• Poor staff 1 

- Terminate 

• Building autonomy needed in better use of software 1 
O and hardware 
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Itartor Question Matrix for Thinking Skills Program (TSP) TMch*r, Principal/Assistant 
Principal, and Studsnt Q— tionnairss Plus TSP Daachar Observation Instnsawt 



MASTER 
QUESTION 
NUMBER 


QUESTIONNAIRE ITEM NtMfeER 
Teacher Principal/Assistant Principal 


OBSERVATION 
Student Teacher 


1. 


1. 


1. 






1. — 


2. 


2. 


2. 






2. — 


3. 


3. 


3. 






3. — 


4. 


4. 


4. 






4. 


5. 


5. 


5. 






5. 


6. 


6. 


6. 






— 


7. 


7. 


7. 






— 


8. 


8./ 


8. 









9, 


9. 


9. 






6. 


10. 


10. 


10. 






— 


11. 


u. 


11. 






— — 


12. 


12. 


12. 






— 


13. 










7. 


14. 










8. 


15. 


13. 


13. 









16. 


14. 


14. 









17. 


15. 


15. 









18. 


16. implemented 


16. difficult to impleraent 





19. 


17. 


17. 






9. 


20. 


18. 


— 









21. 




— 






10. 


22. 


— 








11. 


23. 


19. 


18. 









24. 


20. 


19. 










21. 


20. 








26. 




21. 








27. 


22. 










28. 


23. 








, 


29. 


24. cofflputer/ 
suppl ies 


22. 


supplies 







30. 


25. 










31. 


26. 


24. 








32. 
33. 


points 
28. weak points 


25. 
26. 


strong points 
weak points 




12. good points — 
ij • aon t iiKe 


34. 


29. 


7.1. 








35. 


30. 


28. 








36. 


31. 


23. 








37. 





29. 


continue 




14 ♦ take another ■ 1 ■ 
class 


38. 










15. . 


39. 











16. — 


40. 










— 1-1 


41. 










— 1-2 


42. 











— r-3 


43. 










— 1-4 


44. 










— 1-5 


45. 










— 1-6 


46. 










— [-7 


47. 








7° 


— 1-8 

— r-9 


* • 










— n-i 


50. 
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Table G-l 

Mean Rating on Five Point Scale (Strongly Agree = 1, Agree = 2, Unsure = 3, 
Disagree 3 4, and Strongly Disagree ■ 5) of Teachers, Principals/Assistant 
Principals, and Students 



Statement 


Teacher 


Princ ipal/Ass t . Principal 


Student 


1. 


The classrooms have enough 
computers. 


2.42 


1.62 


1.73 


2. 


As TSP units are almost cou- 
ple ted most students begin 
discussion easily. 


1.40 


2.38 


2.24 


3. 


TSP activities and materials 
supplement what the students 
are taught in other classes. 


2.60 


2.25 


2.10^ 


4. 


Having the students write and 
answer eight questions works 
to stimulate learning. 


2.66 


2.12 


2.39 


5. 


Displaying the students 1 work 
stimulates learning. 


1.73 


1.50 


2.28 


6. 


The inservice training in TSP 
methods was helpful for both 
the TSP and regular education 
teachers. 


1.86 a 


2.00 




7. 


The regular education teachers 
are cooperative with TSP 
teachers . 


1.85 


2.00 




8. 


Students can write more complete 
questions and answers as a 
result of being in a TSP class. 


2.50 


2.50 




9. 


Students who do good enough at 
the end of their first year 
should be taken out of the 
program. 


2.78 


2.25 


o r nb 

2.63 


10. 


Students are interested and 
motivated by TSP activities 
and materials. 


1.80 


1.50 




11. 


TSP teacher morale and interest 
in the program is high. 


2.06 


1.75 




12. 


multiple proW em-solving tech- 
niques (for example , verbal 
deductive reasonirg, etc.). 


1.53 


2.25 


— 


13. 


My teacher is comfortable 
with the TSP materials and 
computers. 






1.82 


14. 


My teacher encourages re to 
think about a problem in 
different ways. 






1.70 

i 



Note. N = 508. Teachers N = 15, Princ ipals/Asst. Principals N = 8, and N = 495 Students. 

a For compensatory education teachers this item was worded just for them specifically, 
i.e., "The inservice training in TP methods was helpful." 

b For students this item was worded in the reverse, i.e., "If I do good enough at the 
end of my first year, I should be allowed to stay in the program. " 
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1993-94 TSP CLASSROOM OBSERVATION INSTRUMENT 

(N = 18) 



INSTRUCTIONS: Please check or fill-in as appropriate. 



Enrollment: 


Mean = 


■ 13 


Limits 


of 


range 


= 6 


to 


16 




Mode = 


13 
















Median = 


■ 13 














Attendance: 


Mean = 


■ 10 


Limits 


of 


range 


= 4 


to 


14 




Mode = 


= 9 
















Median : 


= 10 















I. CLASSROOM ENVIRONMENT 

1. What is the number of computers which are in gocd working condition? 

Mean = 13 Limits of range = 11 to 17 

Mode = 13 

Median = 13 

2. How many samples of student work are displayed on the walls? 

Mean = 6.6 Limits of range = 0 to 37 

Mode = 0 
Median = 0 

3. How is attendance taken? 

Informal procedure = 13 (72.2) 
Formal procedure* = 5 (27.3) 

* One classroom took oral calling of roll which took 5 minutes 

A. Does the teacher encoura^fe students to solve problems with "thinking strategies"? 

Yes 17 (94.4) 

No 1 ( 5.6) 

5. Do the students seem motivated and/or enthusiastic toward the class activities? 

Yes 17 (94.4) 

No 1 ( 5.6) 

6. Does the teacher seem motivated and/or enthusiastic toward the class activities? 

Yes 17 (94.4) 

No 1 ( 5.6) 
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7. Does the teacher seem to be able to motivate and/or encourage students? 

Yes 17 (94.4) 

No 1 ( 5.6) 



8. Does t! teacher refer to class work in the regular classroom and show how its 
related to TSP class work? 

Yes 3 (16.7) 

• No 15 (83.3) 



9. Is the class reaching the end of a unit? 

Yes 1 ( 5.6) 

No 17 (94.4) 

If yes, are students writing a group of questions? 

Yes 0 ( 0.0) 

No .1 (100.0) 



7e 

69 



APPENDIX H 



II. INSTRUCTION 

1. What is the major learning activity during each ten minutes of the TSP class 
and what role does the teacher play in that activity? 



£ctivity/Fole 




i 




ien- 


-Minute Period 
3 


A 




D 


Question and/or lecture/presenter 


13( 


72.2) 


6( 


33.3) 


2( 


11.1) 


K 


5.6) 


2( 


11.1) 


Machine opera t ion /helper 


0( 


0.0) 


6( 


33.3) 


9( 


50.0) 


10( 


55.6) 


9( 


50.0) 


Sxjinmary/questioner 


0( 


0.0) 


0( 


0.0) 


0( 


0.0) 


0( 


0.0) 


K 


5.6) 


Review/question and lecture 


2( 


11.1) 


2( 


11.1) 


0( 


0.0) 


0( 


0.0) 


0( 


0.0) 


Machine demonstration/modeling 


0( 


0.0) 


0( 


0.0) 


2( 


11.1) 


0( 


0.0) 


0( 


0.0) 


Board and/or seat vork/supervisor 


K 


5.6) 


3( 


16.7) 


4( 


22.2) 


6( 


33.3) 


5( 


27.8) 


Testing/tester 


K 


5.6) 


0( 


0.0) 


0( 


0.0) 


0( 


0.0) 


0( 


0.0) 


Group reading and recording/monitor 


K 


5.6) 


K 


5.6) 


K 


5.6) 


K 


5.6) 


K 


5.6) 


TOTAL 


18(100.1)* 


18(100.0) 


18(100.0) 


18(100.1)* 


18(100.1)* 



*Due to rounding. 



2. What type of lesson was presented? (Please provide descriptive details.) 



Book activity (Huck Finn, VJhen a Hero Dies, Terabithia, etc.) 

Patterns 

Game Show 

Carmen San Diego 

tferthwood Mventure 

Valentine card 

Letterhead 

Long division using decimals 
Multiplication/division of one or two 
Story solver 

Current events position statement 
Guessing versus prediction 



(16.7) 
(11.1) 
(11.1) 
(11.1) 
(11.1) 
( 5.6) 
( 5.6) 
( 5.6) 
( 5.6) 
( 5.6) 
( 5.6) 
( 5.6) 



70 



77 



BIBLIOGRAPHY 



Banks, J. R. Selecting a Thinking Skills Program , Lancaster, PA: 
Technanic Publishing. 

Gore, K. (1991). DISKovery: Conputers and Thinking Skills: The HOTS 
Program. Language Arts , 68 , 153-158. 

Pogrcw, S. (1990). Challenging At-Risk Students: Findings from the 
HOTS Program. Phi Delta Kappan , 71, 389-397. 

Pogrow, S. (1993). Where's the Beef? Looking for exemplary materials. 
Educational Leadership , 51 , 39-45. 



78 

71 



